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Four  methods  were  examined  for  speed  and  accuracy  in  inputting 
tactical  messages  concerning  enemy  activity  into  an  Army  computer  for- 
mat. The  methods  were  (a)  typing--the  user  types  the  appropriate  codes 
into  a message  format;  (b)  typing  with  an  error  corrector — the  computer 
automatically  attempts  to  correct  common  spelling  and/or  typing  errors; 
(c)  menus — the  user  indicates  which  of  the  legal  entries  is  desired  from 
a list;  and  (d)  typing  with  autocompletion  and  an  English  option — the 
user  must  type  only  sufficient  characters  to  uniquely  identify  the  item, 
using  either  the  appropriate  code  or  its  English  definition. 


Thirty-two  enlisted  persons  representative  of  the  class  of  military 
personnel  who  might  serve  as  input  device  operators  input  one  of  four  sets 
of  nine  messages  using  each  inputting  method. 


The  use  of  menus  was  ti  ? most  accurate  inputting  method.  For  users 
of  limited  experience  (1  day  of  inputting) , there  were  no  differences  in 
speed  among  the  inputting  methods.^ 

Consideration  should  be  given  to  the  adoption  of  menus  in  tactical 
operation  systems  with  a menu  override  option  for  experienced  users. 

The  use  of  an  error  corrector,  autocompletion,  or  an  English  option 
is  probably  not  warranted  unless  operational  use  shows  a specific  need 
for  such  an  aid. 
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FOREWORD 


The  Battlefield  Information  Systems  Technical  Area  of  the  Army  Re- 
search Institute  (ARI)  is  concerned  with  the  demands  of  increasingly 
complex  battlefield  systems  that  are  used  to  acquire,  transmit,  process, 
disseminate,  and  utilize  information.  This  increased  complexity  places 
greater  demands  upon  the  operator  interacting  with  the  machine  system. 
Research  in  this  area  is  focused  on  human  performance  problems  related 
to  interactions  within  command  and  control  centers  as  well  as  on  issues 
of  systems  development.  Such  research  is  concerned  with  software  devel- 
opment, topographic  products  and  procedures,  tactical  symbology,  user- 
oriented  systems,  information  management,  staff  operations  and  procedures, 
decision  support,  and  sensor  systems  integration  and  utilization. 

An  issue  of  special  concern  within  the  area  of  user-oriented  systems 
is  the  improvement  of  manual  data  input  procedures,  especially  in  the 
Tactical  Operations  System  (TOS) . The  main  source  of  information  for 
tactical  data  systems  is  manual  data  entry--a  slow,  error-prone  process. 
The  capability  of  tactical  data  systems  such  as  TOS  to  support  command 
staff  actions  with  accurate,  complete,  and  timely  information  is  depen- 
dent on  the  performance  of  the  person  who  must  manually  enter  information 
into  the  system.  Previous  ARI  research  on  data  entry  has  resulted  in 
simplified  message  formats,  improved  reference  codes,  and  aids  for  on- 
line preparation  and  verification  of  message  entries.  Although  progress 
has  been  made,  data  entry  remains  a major  system  bottleneck.  The  research 
reported  here  compared  alternative  inputting  methods  with  four  levels  of 
system  aiding  and  prompting  and  has  produced  specific  design  recommenda- 
tions for  improving  the  speed  of  data  entry  and  error  rates. 

Research  in  user-oriented  systems  is  conducted  as  an  in-house  ef- 
fort augmented  through  contracts.  This  report  resulted  fron  an  in-house 
research  effort  responsive  to  requirements  of  Army  Project  2Q763743A/74 
and  to  special  requirements  of  the  U.S.  Army  Combined  Arms  Combat  Devel- 
opment Activity,  Fort  Leti venworth , Kans . Special  requirements  are  con- 
tained in  Human  Resource  Need  76-162  (77-295) , "Processor  Aided  Retrieval 
and  Storage." 
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BRIEF 


Requirement : 

Nearly  all  information  in  tactical  operations  systems  is  input 
manually.  Two  problems  that  arise  when  manually  inputting  data  are 
(a)  the  introduction  of  errors  in  translating  information  into  computer 
format  and  (b)  the  introduction  of  a bottleneck  in  total  system  response 
time.  Therefore,  alternative  methods  of  inputting  data  for  accuracy  and 
speed  should  be  evaluated. 


Procedure : 

Four  methods  were  examined  for  speed  accuracy  when  inputting  tacti- 
cal messages  concerning  enemy  activity  into  an  Army  computer  format. 

The  methods  were  (a)  typing — the  user  types  the  appropriate  codes  into 
a message  format;  (b)  typing  with  an  error  corrector — the  computer  auto- 
matically attempts  to  correct  common  spelling  and/or  typing  errors;  (c) 
menus — the  user  indicates  which  of  the  legal  entries  is  desired  from  a 
list;  and  (d)  typing  with  autocompletion  and  an  English  option — the 
user  must  type  only  sufficient  characters  to  uniquely  identify  the 

item,  using  either  the  appropriate  code  or  its  English  definition. 

> 

Thirty-two  enlisted  persons,  representative  of  the  class  of  mili- 
tary personnel  who  might  serve  as  input  device  operators,  input  one  of 
four  sets  of  nine  messages  using  each  inputting  method. 

Findings : 

The  use  of  menus  was  the  most  accurate  inputting  method.  For  users 
of  limited  experience  (1  day  of  inputting),  there  were  no  differences  in 
speed  among  the  inputting  methods. 

Utilization  of  Findings: 

Consideration  should  be  given  to  the  adoption  of  menus  in  tactical 
operation  systems  with  a menu  override  option  for  experienced  users. 

The  use  of  an  error  corrector,  autocompletion,  or  an  English  option  is 
probably  not  warranted  unless  operational  use  shows  a specific  need  for 
such  an  aid. 
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TACTICAL  DATA  INPUTTING 


INTRODUCTION 

Despite  the  long-term  possibility  of  direct-sensor  to  computer- 
data  transfer,  the  main  source  of  information  for  tactical  operations 
systems  is  manual  data  inputting.  Two  problems  arise  when  data  are  en- 
tered into  the  system  manually:  (a)  the  introduction  of  errors  in  trans- 
lating information  into  computer  format  and  (b)  the  introduction  of  a 
bottleneck  in  total  system  response  time. 

The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI)  has  been  engaged  in  research  to  improve  performance  of 
data  inputting  at  the  man-machine  interface  (see  Alderman  (1976)  for 
a discussion  of  previous  research) . Earlier  research  sought  to  im- 
prove human  performance  by  individualized  training  techniques  (Gade, 
Fields,  & Alderman,  1978),  by  computer  prompting  and  instruction  (Strub, 
1975) , and  by  using  on-line  inputting  with  verification  (Strub,  1971)  . 

The  earlier  research,  however,  did  not  focus  on  actual  inputting 
procedures.  To  learn  how  the  method  of  inputting  affects  speed  and 
accuracy,  four  different  methods  for  entering  intelligence  data  into 
a Tactical  Operating  System  (TOS)  format  were  examined.  The  fastest 
method  could  be  used  to  alleviate  the  bottleneck  at  the  man-machine 
interface.  The  problem  of  introduction  of  error  has  two  parts:  (a) 
the  operator  must  form  a correct  concept  of  the  information  (from  read- 
ing a message,  hearing  a telephone  message,  viewing  a radar  scope,  etc.) 
and  (b)  the  operator  must  input  properly  formatted  information  based  on 
a correct  concept.  In  other  words,  an  error  could  arise  if  the  opera- 
tor misunderstood  the  information  or  incorrectly  input  the  information. 
Although  it  is  difficult  to  separate  the  two  sources  of  error,  the 
method  of  inputting  affects  inputting  errors  rather  than  concept  forma- 
tion errors. 

The  first  of  the  four  methods  chosen  for  testing  was  typing  in 
which  the  user  types  the  appropriate  codes  into  a message  format.  Typ- 
ing is  a common  method  of  inputting  and  is  used  in  most  versions  of  the 
Army's  developing  Tactical  Operations  System  and  previous  ARI  research. 

The  second  inputting  method  was  also  typing,  except  that  a computer 
automatically  attempted  to  correct  common  spelling  and  typing  errors. 

The  computer  should  be  able  to  make  corrections  faster  than  the  human 
user  and  thus  speed  up  the  rate  of  inputting. 

A third  inputting  method  used  menus,  in  which  the  user  indicates 
which  of  the  legal  entries  listed  is  desired.  A track  ball  moving  a 
cursor  was  used  to  indicate  the  entry.  This  method  is  a common  alter- 
native to  typing;  it  requires  less  training  and  the  user  cannot  make 
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spelling  errors.  Also,  because  the  use  of  menus  places  a smaller  cog- 
nitive load  on  the  user  (requiring  recognition  memory  rather  than  recall 
memory),-^  there  is  also  the  possibility  that  fewer  errors  will  occur. 

The  fourth  method  of  .'nputting  was  aimed  especially  at  speeding  up 
the  rate  of  inputting.  This  method,  which  used  typing  as  its  base,  re- 
quired that  the  user  type  only  enough  characters  to  identify  a member 
of  the  table  of  legal  entries  for  that  item  of  the  format.  The  computer 
would  fill  in  the  rest  of  the  entry,  thus  saving  typing  time.  Because 
research  has  sh  wn  that  typing  English  is  faster  than  typing  codes,  the 
user  of  this  method  was  allowed  to  use  either  the  code  or  its  English 
definition.  In  either  case,  the  participant  was  to  use  only  the  mini- 
mum number  of  characters  needed  to  identify  the  code  or  definition. 

The  different  operating  characteristics  of  the  four  methods  pro- 
vide a basis  for  predicting  relative  inputting  performance.  Thus,  com- 
pared with  normal  typing,  typing  with  an  error  corrector  should  result 
in  fewer  errors  as  well  as  a time  savings.  Similarly,  the  use  of  menus 
should  result  in  fewer  errors  and  possibly  a time  savings.  However,  the 
fastest  and  most  accurate  inputting  method  should  be  autocompletion. 


OBJECTIVE 

The  purpose  of  this  research  was  to  evaluate  alternative  methods  of 
inputting  enemy  situation  data  for  speed,  accuracy,  and  ease  of  use. 


METHOD 

Subjects 

Thirty-two  enlisted  men  and  women  were  randomly  assigned  to  four 
groups  for  the  experiment.  Participants  possessed  a GT  score  of  105  or 
above,  vision  correctable  to  20/20,  and  were  representative  of  the  class 
of  military  personnel  who  might  be  trained  to  serve  as  input  device  oper- 
ators. Although  26  of  the  32  participants  claimed  some  typing  skill,  and 
several  had  had  limited  exposure  to  computers,  none  had  used  the  inputting 
techniques  being  evaluated  in  the  experiment. 


References  to  this  phenomenon  are  common.  For  examples,  see  Loftus  & 
Loftus,  1976,  pp.  56-91,  or  Baddeley,  1976,  pp.  285-286. 
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Task 


The  participants  acted  as  intelligence  staff  members  entering  in- 
formation about  the  enemy  into  a computerized  data  base.  They  received 
free  text  messages  describing  enemy  actions.  An  example  appears  in 
Figure  1.  Their  job  was  to  extract  the  necessary  information  and  enter 
the  appropriate  codes  into  a variation  of  the  Enemy  Situation  Data  Add 
(ESDA)  format  of  the  TOS  system  (see  Figure  2) . 


Experimental  Station 

The  experiment  was  conducted  in  an  area  containing  a table  and  chair 
for  the  participant.  Two  cathode  ray  tube  (CRT)  displays  sat  on  the 
table,  one  showing  the  format  into  which  the  participant  would  enter 
data  and  the  other  containing  the  menus  for  the  menu-selection  inputting 
method.  A track  ball  for  the  menu-selection  method  was  placed  in  front 
of  that  CRT.  A keyboard  was  placed  in  front  of  the  format  CRT.  A dic- 
tionary was  available  that  contained  the  valid  codes  and  their  English 
definitions  cross-indexed  to  the  items  in  the  format.  An  intercom  was 
provided  for  calling  the  experimenter. 


Inputting  Methods 

Four  different  inputting  methods  were  investigated  in  this  experi- 
ment. Each  method  used  automatic  tabbing;  that  is,  after  the  completion 
of  an  entry,  the  computer  automatically  moved  the  cursor  to  the  next  en- 
try item  in  the  format.  The  computer  did  not  accept  invalid  codes^  for 
an  entry  and  gave  the  user  an  error  message  to  that  effect. 


I . Typing . In  this  method,  the  message  format  appeared  on  the 
screen  with  the  cursor  placed  at  the  first  item.  The  users  then  filled 
in  the  appropriate  codes  (including  blanks)  for  each  item,  referring  to 
the  dictionary  of  valid  codes  and  their  definitions  as  needed.  The  com- 
puter rejected  any  input  that  was  not  a legal  entry  and  would  not  allow 
the  user  to  continue  until  a valid  code  was  entered.  Participants  could 
backspace  and  type  over  their  answers.  The  cursor  automatically  moved 
to  the  next  item  in  the  message  format  when  a valid  entry  had  been  typed 


An  invalid  code  is  one  that  does  not  appear  in  the  valid  code  list  for 
an  entry.  The  terms  "valid"  and  "legal"  are  used  interchangeably.  A 
valid  code  may  be  incorrect  if  it  appears  in  the  valid  code  list  but  does 
not  correctly  describe  the  information  in  the  free  text  message. 


3 
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SPOT  REPORT 

PRECEDENCE : 

DISPOSITION 


^FLASH^ 


GI 


ENGR 


FROM:  1st  Brigade, 


- IMMEDIATE  - PRIORITY 


- ROUTINE 

FSE  TASE 


3BDE 


Ground  Surveillance  Section 


REPORT  TIME:  1745 

EVENT  TIME:  1630 


CLASSIFICATION:  (u)  C 


S TS 


DESCRIPTION:  Convoy  moving  SW  along  Route  #15 
7 tanks,  22  trucks,  52  APC's 
picked  up  by  unattended  ground  sensors 
vie,  XT437262 


MESSAGE  NUMBER  - 45618 


Figure  1.  Typical  free  text  message  describing  enemy  actions. 
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ENEMY  SITUATION  DATA  ADD  - ESDA 


ORIG/NO : [ / ] SCTY [ ] PRES [ ] RESTR[  ] 

RECD-FR[  ] REPT-ID  [ ] REL-TO [ ] 

AGENCY [ ] SOURCE-ID [ ] 

EVENT-TIME [ ] NATION l ] 


EN-UNIT [ / / / ]EN-PARENT[  / II] 

— SUBJ ACTV QUANT-DESCRIPTION LOCATION- 

u / / / / > 

2<  / / / / ) 

3(  / / / / ) 

4(  / / / / ) 

5<  / / / / ) 

REMARKS [ ] 

DISTR[  , , , , ] INTEREST!  ] 


Figure  2.  Format  for  entering  data. 
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II.  Typing  with  Error  Correction.  This  method  was  the  same  as 
Method  I except  that  an  aid  had  been  added — a typing  error  corrector. 

When  the  user  typed  an  invalid  entry,  the  system  assumed  that  a typing 
error  had  been  made  and  attempted  to  correct  the  error.  The  typing 
error  corrector  dealt  with  four  kinds  of  errors:  transposition  of  two 
adjacent  letters,  insertion  of  an  extra  letter,  deletion  of  one  letter, 
or  substitution  of  one  incorrect  letter.  When  the  typed  item  was  an 
invalid  entry,  the  typing  corrector  tried  to  form  a hypothesis  of  '.■.hat 
the  entry  should  be.  For  each  of  the  four  kinds  of  errors,  the  typing 
corrector  found  a list  of  ail  the  members  of  the  valid  list  that  were 
either  the  same  length  as  the  invalid  entry,  one  character  shorter,  or 
one  character  longer,  depending  on  which  of  the  four  types  of  typing 
errors  was  being  tested.  Then  it  went  through  one  of  four  processes: 

1.  The  typing  corrector  scanned  the  target  word  and  the  entry  from 
the  left  for  mismatches.  At  the  first  mismatch,  it  transposed  the  mis- 
matched letter  .in  the  entry  with  the  letter  to  the.  right.  If  the  new 
word  matched  the  target  word,  the  target  word  was  the  hypothesis.  If 
there  was  no  match,  the  typing  corrector  went  on  to  the  next  target  word. 

2.  The  typing  corrector  scanned  the  target  word  and  the  entry  from 
the  left  for  mismatches.  It  deleted  the  mismatched  letter  in  the  entry 
and  checked  for  a match.  If  there  was  no  match,  the  typing  corrector 
went  on  to  the  next  target  word. 

3.  The  typing  corrector  scanned  from  the  left  for  mismatches  and 
deleted  the  mismatched  letter  from  the  target  word  and  checked  for  a 
match.  If  there  was  no  match,  it  went  on  to  the  next  target  word. 

4.  The  typing  corrector  scanned  from  the  left  for  mismatches.  At 
the  first  mismatch,  it  skipped  the  mismatched  letters  in  both  words  and 
checked  the  rest  for  a match.  If  there  was  no  match,  the  typing  correc- 
tor went  on  to  the  next  target  word . 

The  first  hypothesis  formed  when  the  typing  corrector  found  a match 
printed  out  on  the  screen  with  an  error  message.  If  it  was  the  entry 
the  user  actually  wanted,  the  user  hit  a key  to  signal  acceptance:  other- 
wise the  user  could  hit  a key  to  signal  a retyping  of  the  entry.  If  the 
typing  corrector  could  not  form  a hypothesis,  an  error  message  appeared 
on  the  screen  and  the  user  had  to  retype  the  entry. 

III.  Menus.  In  this  input  method,  typing  was  used  for  entering 
three  types  of  items  in  the  message  format:  map  coordinates,  dates,  and 
cardinal  numbers.  All  other  data  .had  to  be  entered  by  selecting  the  ap- 
propriate item  from  an  alphabetically  or  logically  ordered  menu;  data 
could  not  be  typed  in.  The  message  format  appeared  on  "the  right  CRT  with 
a cursor  at  the  first  item.  The  menu  of  legal  entries  for  ’fcbie  first  item 
appeared  on  the  left  CRT  in  their  unccded  form  (i.e.,  English  definitions 
from  the  dictionary) . If  a blank  was  a valid  entry,  it  was  included  in 
the  menu.  When  an  item  was  chosen,  its  code  was  filled  into  the  message 
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format  by  the  computer  and  the  cursor  moved  to  the  next  item.  If  an  item 
had  to  be  typed  rather  than  chosen  from  a menu  (e.g.,  map  coordinate 
string) , instructions  to  type  the  item  would  be  displayed  where  the  menu 
would  normally  appear.  In  these  cases,  invalid  entries  (e.g.,  insuffi- 
cient characters  in  the  coordinate  string)  were  rejected  and  the  next 
item  could  not  be  entered  until  a valid  entry  had  been  typed  in,  as  in 
Method  I. 

IV.  Typing  with  Autocompletion  and  English  Option.  This  method 
was  the  same  as  Method  I,  but  with  two  additions.  First,  the  partici- 
pant could  enter  the  English  definition  from  the  dictionary  in  place  of 
the  code,  as  desired.  Second,  when  the  participant  felt  that  enough 
characters  had  been  typed  to  identify  a member  of  the  valid  entry  list 
(either  code  or  English) , he  would  push  the  send  button,  causing  the 
autocompletion  program  to  take  over.  If  indeed  the  program  could  match 
the  characters  with  the  beginning  of  one,  and  only  one,  member  of  the 
valid  entry  list,  it  would  automatically  finish  the  entry  for  the  parti- 
cipant and  move  the  cursor  to  the  next  entry.  If  a unique  match  could 
not  be  made,  the  program  asked  for  more  characters.  The  participant 
could  invoke  the  autocompletion  program  at  any  time  and  as  often  as 
wished  until  a unique  match  was  made. 


Dependent  Variables 

The  primary  dependent  variables  were  format  completion  time  and 
accuracy  in  terms  of  number  of  errors  per  format.  Typing  errors  as  well 
as  errors  of  interpretation  and  errors  specific  to  a particular  inputting 
method  were  included  in  evaluating  the  number  of  errors  per  format. 

Other  data  collected  included  the  number  of  times  the  typing  correc- 
tor was  used,  the  number  of  times  the  typing  corrector  correctly  identi- 
fied the  target  code,  the  number  of  times  autocompletion  was  used,  the 
number  of  times  English  definitions  were  used  in  place  of  codes,  the 
number  of  times  the  participant  corrected  an  entry  by  backspacing  and 
typing  over  before  entering  the  item,  and  the  participant's  stated 
preferences  among  the  input  methods.  All  the  participants'  entries 
and  the  entry  times  were  recorded  as  well. 


PROCEDURE 

Thirty-six  free  text  messages  describing  enemy  actions  were  divided 
into  four  sets  of  nine  messages  with  each  message  set  (M)  balanced  for 
types  of  subjects,  sources,  restrictions,  unit  identifications,  and 
difficulty  during  the  pilot  testing.  For  each  message  set,  the  first 
message  was  always  a practice  message,  and  the  other  eight  messages 
made  up  the  experimental  set.  In  addition  to  these  36  messages,  2 other 
messages  were  initially  presented  as  practice  messages. 
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RESULTS 


Analysis  of  variance  summary  tables  and  means  for  the  dependent 
variables  in  this  experiment  can  be  found  in  the  appendix.  Only  major 
findings  are  given  here. 


Accuracy 

Accuracy  of  an  inputting  method  may  be  considered  from  two  perspec- 
tives: (a)  mean  errors  per  message  and  (b)  the  number  of  participants 

who  had  the  fewest  errors  using  a particular  method.  For  both  measures, 
the  rank  order  of  inputting  methods  was  the  same  (see  Table  2) . 


Table  2 

Accuracy  Ranking  of  Inputting  Methods 


Method 

Mean  number  of 
errors  per  message 

Participants  who  made 
the  fewest  errors 
using  this  method 

Menus 

2.64 

17 

Typing  with  error 

corrector 

3.36 

8.5 

Typing 

3.77 

3.5a 

Typing  with 

autocompletion 

4.39 

3 

Includes  one  tie  between  typing  and  typing  with  error  corrector. 


An  analysis  of  variance  of  the  mean  error  data  indicates  that  the 
differences  among  the  inputting  methods  are  significant  (F  = 13.98, 
df  = 3,84,  p < .001).  There  were  no  statistically  significant  residual 
interactions  between  inputting  methods  and  message  set,  sessions,  or 
groups.  The  analysis  of  variance  table  (Table  A-l)  and  a discussion 
of  other  significant  findings  is  provided  in  the  appendix. 
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Time 


The  mean  time  per  message  using  each  of  the  four  inputting  methods 
and  number  of  participants  who  had  their  fastest  scores  using  a given 
inputting  method  are  shown  in  Table  3. 


Table  3 

Time  Ranking  of  Inputting  Methods 


Method 

Mean  time  per 
message  (in  seconds) 

Number  of  participants  who 
averaged  the  fastest  time 
using  this  method 

Typing  with 

autocompletion 

413.13 

13 

Typing  with  error 

corrector 

397.01 

9 

Typing 

396.05 

7 

Menus 

396.52 

3 

Although  scoring  by  number  of  participants  (who  averaged  their  best 
times  over  the  eight  messages  using  a given  inputting  method)  is  consis- 
tent with  the  original  predictions  (see  the  discussion  of  inputting  methods 
in  the  Introduction) , the  mean  time  ranking  is  almost  the  exact  reverse. 

An  analysis  of  variance  perfoimed  on  both  the  raw  time  scores  and  the 
logs  of  the  time  scores  (used  because  of  skew  in  the  time  scores)  showed 
that  the  differences  among  inputting  methods  in  mean  times  were  statis- 
tically insignificant.  (The  analysis  of  variance  summary  tables  can  be 
found  in  Tables  A-2  and  A-3  of  the  appendix.)  The  most  significant  factor 
in  the  analysis  of  variance  of  the  time  data  was  Sessions.  The  more  prac- 
tice a participant  had,  the  shorter  the  inputting  time  became  (p  < .001) . 
This  can  be  seen  in  the  mean  times  across  all  inputting  methods  for  each 
session,  shown  in  Table  4. 
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Table  4 


Time  Data  for  Sessions 


Session 


Mean  time  (in  seconds) 


1 531.11 

2 387.12 


3 


360.21 


4 


324.27 


Preferences 


As  the  participants  were  debriefed,  they  were  asked  about  their 
preferences  among  inputting  methods.  It  was  hypothesized  that  people 
would  prefer  a method  with  which  they  had  performed  well — either  in 
making  the  fewest  errors  or  in  averaging  the  fastest  time.  The  prefer-^ 
ence  results  can  be  seen  in  Tables  5 and  6.  A coefficient  of  agreement 
(K)  and  an  approximation  of  the  standard  error  (OK)  were  determined  for 
each  table.  (See  Cohen,  1960,  for  a discussion  of  K.)  Contrary  to  ex- 
pectations, there  is  chance  agreement  or,  at  best,  a slight  negative 
agreement  between  preference  and  best  performance.  For  fewest  errors, 

< = -.108  and  ok  = .149.  For  fastest  time,  K = -.128  and  OK  = .057. 


Use  of  Computer  Aids 

Backspacing . Whe  . ver  a participant  was  typing  an  entry,  the  back- 
spacing option  was  available.  This  option  allowed  the  subject  to  back  up 
and  type  over  an  answer  to  change  it.  The  use  of  the  backspace  option 
allowed  the  participant  to  catch  and  correct  mistakes  before  the  computer 


k,  the  coefficient  of  agreement,  is  a measure  of  the  agreement  between 
two  variables  measured  by  nominal  scales  (in  this  case,  best  performance 
and  preference) . Like  the  Pearson  product-moment  correlation  coefficient, 
r,  k can  take  on  va  Ivies  only  between  -l  and  +1.  A score  of  +1  would 
indicate  total  agreement  between  per tormance  and  preference,  and  a score 
of  -1  would  indicate  that  users  never  preferred  the  inputting  method  they 
did  best  on.  A score  of  0 would  indicate  no  predictive  linear  relation- 
ship between  preference  and  performance. 
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Table  5 


Agreement  Table  Between  Preferences  and 
Fewest  Errors,  by  Inputting  Method 


Fewest  errors 

Preference 

Typing 

Error 

corrector 

Menus 

Autocompletion 

Total 

Typing 

0 

0 

2 

0 

2 

Error 

corrector 

2 

0 

5 

1 

8 

Menus 

1 

3 

10 

3 

17 

Autocompletion 

1 

0 

1 

0 

2 

Total 

4 

3 

18 

4 

293 

a 

Three  subjects 

did  not 

express  preferences. 

Table  6 

Agreement  Table  Between  Preferences 
Best  Time,  by  Inputting  Method 

and 

Best  time 

Preference 

Typing 

Error 

corrector 

Menus 

Autocompletion 

Total 

Typing 

0 

0 

5 

0 

5 

Error 

corrector 

1 

1 

5 

2 

9 

Menus 

0 

2 

0 

1 

3 

Autocompletion 

3 

0 

8 

1 

12 

Total 

4 

3 

18 

4 

293 

Three  subjects  did  not  express  preferences. 
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would  catch  them.  This  option  was  frequently  used  by  the  participant, 
as  shown  in  Table  7.  The  backspacing  option  could  be  used  with  three 
results:  (a)  a correct  entry,  (b)  a valid  (correctly  spelled)  entry 

that  was  not  the  correct  entry  for  the  message,  or  (c)  an  invalid  entry. 

Error  Corrector.  Table  8 shows  the  use  of  the  error  corrector. 

The  error  corrector  could  change  an  invalid  entry  to  a valid  and  cor- 
rect entry,  and  could  change  an  invalid  entry  to  a valid  and  correct 
entry  but  the  participant  could  reject  that  correct  answer.  In  about 
a third  of  the  cases  the  error  corrector  could  not  arrive  at  a valid 
entry . 

Autocompletion  with  English  Option.  Table  9 shows  the  use  made  of 
the  autocompletion  option  and  the  English  option.  There  were  six  forms 
these  options  could  take.  The  subject  could  use  the  full  English  defini- 
tion, and  that  definition  could  be  either  correct  or  incorrect.  The 
subject  could  use  autocompletion  on  either  the  code  or  the  English  defi- 
nition and  these  autocompleted  entries  could  be  either  correct  or 
incorrect . 


DISCUSSION 


Menus 

Menus  appear  to  be  more  error-free  than  the  other  inputting  methods 
examined  (see  Table  2) . Indeed,  there  is  a 40%  decrease  in  error  with 
menus  from  the  worst  case,  typing  with  autocompletion.  Menus  are  popu- 
lar; 18  out  of  29  participants  expressed  a preference  for  them.  Finally, 
menus  do  not  have  higher  mean  times  than  the  other  inputting  methods. 
Therefore  it  would  seem  reasonable  to  suggest  that  the  use  of  menus  be 
seriously  considered  for  adoption  for  use  in  tactical  data  inputting. 

This  study  considered  menu  selection  with  a track  ball.  However, 
the  track  ball  method  of  menu  selection  does  not  seem  to  be  a good  method 
of  inputting.  It  is  slow  and  cumbersome  to  use.  There  are  several  other 
methods  of  a menu  selection  (e.g.,  liqhtpen,  touch-sensitive  panels,  a 
typed  index  code  or  letter)  that  should  be  considered.  Each  method  has 
advantages  and  disadvantages;  the  results  of  this  research  do  not  suggest 
which  one  would  be  the  best  for  the  tactical  data  inputting  task. 

None  of  the  menus  in  this  research  was  particularly  long.  Most  fit 
on  one  screen  (i.e.,  they  were  less  than  40  items  lonq) . The  longer 
menus  (e.g.,  "subject")  were  divided  into  sections  (e.g.,  "personnel," 
"vehicle,"  etc.).  The  first  screen  shown  : .he  participant  was  a list 
of  sections  from  which  the  subject  could  select  a screen-sized  menu. 

Even  with  the  relatively  fast  speed  of  the  experimental  displays  and 
the  relatively  short  length  of  the  menus,  participants  still  had  to 
wait  for  a menu  to  be  printed  out  so  that  they  could  respond.  With 
slower  equipment,  longer  menus,  or  more  experienced  users,  delays  due 


Use  of  Backspacing 


Use  of  Error  Corrector 
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On  the  average,  each  subject  had  106  opportunities  to  use  these  options. 


to  either  paging  through  a menu  or  waiting  until  the  menu  was  printed 
out  could  be  a serious  defect.  Therefore,  either  a menu  override  op- 
tion should  be  made  available  for  experienced  users,  or  careful  con- 
sideration should  be  given  to  the  design  of  an  operational  menu  system 
taking  these  point's  into  account.  Otherwise  a poorly  designed  menu 
system  could  negate  the  benefits  found  in  this  research. 


Time  Data 

The  time  data  from  this  experiment  suggest  that  the  participants 
never  reached  a plateau  where  practice  effects  no  longer  affected  per- 
formance. If  such  a plateau  had  been  reached,  perhaps  time  differences 
among  the  inputting  groups  would  have  emerged . 

Would  autocompletion,  for  example,  have  been  faster  if  the  subjects 
had  had  more  experience?  It  is  impossible  to  say.  Autocompletion  was 
the  fastest  inputting  method  for  40%  of  the  participants,  or  13  people. 

An  a priori  estimate  assuming  no  differences  among  inputting  methods 
would  lead  us  to  expect  each  inputting  method  to  be  fastest  for  25%  of 
the  participants.  The  mean  time  of  those  13  people,  308.23  seconds,  how- 
ever, was  faster  than  any  other  group,  and  their  use  of  the  autocomple- 
tion option  was  higher  than  the  total  group  average.  This  mean  inputting 
time  was  also  faster  than  the  average  for  any  of  the  four  sessions.  Re- 
call that  the  analysis  of  variance  on  sessions  produced  the  most  signifi- 
cant results,  indicating  a training  effect  (the  fastest  session  was  the 
last  one,  the  fourth,  and  the  mean  was  324.27) . 

Many  subjects  complained  that  autocompletion  was  confusing  and  dif- 
ficult to  understand.  This  confusion  may  have  led  to  the  high  error 
score  and  wide  variance  seen  in  the  autocompletion  data.  It  would  seem 
that  autocompletion  is  a useful  tool  only  for  a sophisticated  user. 
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English  Option 

The  option  to  use  English  was  not  as  popular  as  the  option  to  use 
codes  (see  Table  9) . The  English  option  was  used  most  often  where  the 
code  and  English  definition  were  most  dissimilar.  In  inputting  the 
precedence  codes  (R  = Routine,  P = Priority,  I = Immediate,  and  Z = 
Flash)  for  example,  F with  autocompletion  was  often  used  rather  than  Z. 
This  unpopularity  may  be  because  the  English  required  many  more  charac- 
ters to  be  typed,  on  the  average,  than  the  code  and  also  required  more 
characters  to  uniquely  identify  the  item.  An  English  option  is  probably 
not  very  useful,  particularly  with  well-designed,  user-oriented  codes. 


CONCLUSIONS 

Menus  are  recommended  for  use  in  inputting  tactical  data  because 
they  appear  to  cause  fewer  inputting  errors  without  noticeably  affect- 
ing input  rate  than  the  other  methods  examined  in  this  research.  How- 
ever, to  maximize  the  benefits  of  the  menus,  special  attention  should 
be  given  to  the  design  of  an  operational  menu-based  inputting  system-- 
particularly  the  length  of  menus,  the  speed  with  which  they  can  be  dis- 
played, and  the  manner  in  which  items  are  selected  from  a menu. 

Spelling  correctors  or  autocompletion  should  be  considered  only 
for  experienced  users  and  in  operational  settings  that  specifically 
require  such  aids. 
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APPENDIX 


STATISTICAL  SUMMARY  TABLES 

Table  A-l  contains  the  analysis  of  variance  summary  table  for 
the  error  data;  Table  A-4  contains  the  means  and  error  data  for  each  of 
the  four  variables  and  for  each  cell . 

Table  A-2  contains  a summary  of  the  analysis  of  variance  for  time 
data.  Several  outliers  were  found  in  the  data  (e.g.,  some  subjects  had 
never  used  a typewriter  keyboard  before  and  literally  had  to  "hunt  and 
peck"  for  each  key) . Because  the  presence  of  outliers  can  unduly  in- 
fluence an  analysis  of  variance,  a second  analysis  of  variance  was  run 
using  logs  of  the  time  score  The  results  of  the  second  analysis  are 
summarized  in  Table  A-3.  Table  A-5  contains  the  means  and  time  data 
for  each  of  the  four  variables  and  for  each  cell. 


DISCUSSION 

Error  Data 

Message  sets  (M)  are  a significant  source  of  variance  as  can  be 
seen  in  Tables  A-l  and  A-4.  Groups  (G)  and  sessions  (S)  are  also  some- 
what different.  However,  since  the  residual  (or  interaction)  term  is 
not  significant  and  since  the  experimental  design  is  a Greco-Latin 
square  with  within-sub ject  measures,  these  factors  do  not  affect  the 
main  conclusion  concerning  inputting  methods . 

Time  Data 

Use  of  the  log  transform  to  reduce  the  skew  in  the  time  data  re- 
duced the  size  of  the  residual  term  (Tables  A-2  and  A-3) . However,  the 
ordering  of  the  other  terms  remained  approximately  the  same.  In  both 
analyses,  inputting  methods  (I)  are  not  significant  and  sessions  (S)  are 
very  significant  (p  < .001)  . 

A mistake  in  numbering  two  messages  put  a short  message  from  set  4 
into  set  3 and  a long  message  from  set  3 into  set  4.  Therefore,  message 
set  3 is  about  eight  items  too  short  and  message  set  4 is  eight  items 
too  long.  This  probably  accounts  for  the  difference  in  means  between 
them  (see  Table  A-5) . 


Table  A-l 

Analysis  of  Variance  Table  for  Error  Data  (32  Subjects) 


Source 


of  variation 

SS 

df 

MS 

F 

Between  subjects 

31 

Groups  (G) 

114.38 

3 

38.13 

3.79* 

Subjects/groups  = e^ 

281.51 

28 

10.05 

Within  subjects 

96 

Message  sets  (M) 

34.20 

3 

11.40 

9.34*** 

Inputting  methods  (I) 

51.18 

3 

17.06 

13.98*** 

Sessions  (S) 

13.09 

3 

4.36 

3.57* 

Residual 

1.76 

3 

0.59 

0.48 

Subjects/groups  = e 2 

102.09 

84 

1.22 

Total 

598.21 

127 

Table  A- 2 


Analysis  of  Variance  Table  for  Time  Data  (32  Subjects) 


Source 

of  variation 

SS 

df 

MS 

F 

Between  subjects 

31 

Groups  (G) 

314557.95 

3 

104852.65 

4.86** 

Subjects/groups  = 

el 

603568.31 

28 

21556.01 

Within  subjects 

96 

Message  sets  (M) 

87759.42 

3 

29253.14 

12.23*** 

Inputting  methods 

(I) 

6631.30 

3 

2210.43 

.92 

Sessions 

789543.52 

3 

263181.17 

109.92*** 

Residual 

41975.21 

3 

13991.74 

5.85** 

Subjects/groups  = 

e2 

200989.96 

84 

2392.74 

Total 

2045025.66 

127 

**  p < .01  . 

***p  <' .001  . 


Table  A- 3 

Analysis  of  Variance  Table  for  Logs  of  Time  Data  (32  Subjects) 


Source 

of  variation 

SS 

df 

MS 

F 

Between  subjects 

31 

Groups  (G) 

313915.66 

3 

104638.55 

4.22* 

Subjects/groups  = 

el 

694914.81 

28 

24818.39 

Within  subjects 

96 

Message  sets  (M) 

64248.09 

3 

21416.03 

10.01*** 

Inputting  methods 

(I) 

1440.03 

3 

480.01 

.22 

Sessions  (S) 

781955.53 

3 

260651.84 

121.81*** 

Residual 

23799.95 

3 

7933.32 

3.71* 

Subjects/groups  = 

e2 

179741.9 

84 

2139.8 

Total 

2060015.97 

127 

* p < .05. 

***p  < .001. 


•; 
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Means  and  Time  Data  (in  Seconds)  for  Four  Variables  and  for  Each  Cell 
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